AFRL-SA-WP-SR-2015-0028 


Objective  Versus  Subjective 
Military  Pilot  Selection 
Methods  in  the  United  States 
of  America 


Joe  Wood,  Charles  Shurlow,  and  Jared  Haynes 


December  2015 


Air  Force  Research  Laboratory 
DISTRIBUTION  STATEMENT  A.  71 1th  Human  Performance  Wing 

Approved  for  public  release.  U.S.  Air  Force  School  of  Aerospace  Medicine 

Aerospace  Medicine  Department 
2510  Fifth  St. 

Wright-Patterson  AFB,  OH  45433-7913 


STINFO  COPY 


NOTICE  AND  SIGNATURE  PAGE 


Using  Government  drawings,  specifications,  or  other  data  included  in  this  document  for  any 
purpose  other  than  Government  procurement  does  not  in  any  way  obligate  the  U.S.  Government. 
The  fact  that  the  Government  formulated  or  supplied  the  drawings,  specifications,  or  other  data 
does  not  license  the  holder  or  any  other  person  or  corporation  or  convey  any  rights  or  permission 
to  manufacture,  use,  or  sell  any  patented  invention  that  may  relate  to  them. 

Qualified  requestors  may  obtain  copies  of  this  report  from  the  Defense  Technical  Information 
Center  (DTIC)  (http://www.dtic.min. 


AFRL-SA-WP-SR-20 15-0028  HAS  BEEN  REVIEWED  AND  IS  APPROVED  FOR 
PUBLICATION  IN  ACCORDANCE  WITH  ASSIGNED  DISTRIBUTION  STATEMENT. 


//SIGNATURE// 


//SIGNATURE// 


COL  JOHN  P.  LYNCH 


COL  LAURA  TORRES-REYES 


Chief,  Aerospace  Consult  Service  Division 


Chair,  Aerospace  Medicine  Department 


This  report  is  published  in  the  interest  of  scientific  and  technical  information  exchange,  and  its 
publication  does  not  constitute  the  Government’s  approval  or  disapproval  of  its  ideas  or  findings. 


This  page  intentionally  left  blank. 


TABLE  OF  CONTENTS 


Section  Page 

LIST  OF  FIGURES . ii 

LIST  OF  TABLES . ii 

1.0  SUMMARY . 1 

2.0  INTRODUCTION . 1 

3.0  METHODS . 2 

4.0  RESULTS . 3 

5.0  DISCUSSION . 5 

6.0  CONCLUSION . 6 

7.0  REFERENCES . 6 

LIST  OF  ABBREVIATIONS  AND  ACRONYMS . 8 


DISTRIBUTION  STATEMENT  A.  Approved  for  public  release. 


Cleared,  88PA,  Case  #2014-3717. 


LIST  OF  FIGURES 


Page 

Figure  1 .  Box  plot  of  ANG  and  ROTC  FSIQ  scores . 4 

Figure  2.  Box  plot  of  ANG  and  ROTC  personality  traits . 5 

LIST  OF  TABLES 

Page 

Table  1.  FSIQ  and  Personality  Scores  of  ANG  Pilots,  ROTC  Pilots,  and  Civilians . 4 


ii 


DISTRIBUTION  STATEMENT  A.  Approved  for  public  release. 


Cleared,  88PA,  Case  #2014-3717. 


1.0  SUMMARY 


In  the  U.S.  Air  Force,  pilots  are  chosen  through  varying  processes.  Although  these  pilot 
selection  methods  share  similarities,  Air  National  Guard  (ANG)  has  a  more  subjective  selection 
process  than  active  duty  (AD).  To  compare,  1,801  ANG  and  1 1,616  AD  pilot  applicants  were 
administered  a  computer-based  intelligence  test,  the  Multidimensional  Aptitude  Battery-II ,  and  a 
personality  measure,  the  NEO  Personality  Inventory-Revised.  Both  groups  shared  similar 
personality  traits  characterized  by  emotional  stability,  extraversion,  and  high  work  ethic. 
Intelligence  scores  were  near  the  superior  range  for  ANG  and  AD  pilots.  Although  chosen  by 
diverse  methods,  both  ANG  and  AD  pilots  shared  similar  cognitive  and  personality 
characteristics. 

2.0  INTRODUCTION 

There  is  a  tradition  in  military  aviation  of  attempting  to  select  only  those  candidates  with 
the  “right  stuff,”  often  defined  as  high  intelligence,  emotional  maturity,  achievement-orientation, 
good  interpersonal  skills,  and  strong  motivation  to  fly  [1].  During  the  infancy  of  aviation, 
however,  standards  were  evolving  and  little  regard  was  paid  to  an  aviator’s  intelligence  or 
personality  characteristics.  Standards  that  were  used  focused  on  screening  out  undesirable 
medical  conditions,  as  many  early  aviators  were  killed  due  to  physical  defects.  In  response,  flight 
surgeons  began  to  impose  more  stringent  medical  standards  for  flyers. 

Mental  health  was  not  widely  considered  in  screening  applicants  until  the  1920s.  This 
change  occurred  after  the  British  found  that  up  to  50%  of  pilot  candidates  during  World  War  I 
developed  a  neurosis  (i.e.,  mental  health  disorder)  prior  to  graduating  flight  training  [2].  This 
training  pipeline  was  both  costly  and  inefficient,  and  flight  surgeons  began  to  consider 
intellectual  and  personality  characteristics  of  pilots  with  more  zeal.  In  1928,  Rhoades,  a  U.S. 
Navy  flight  surgeon,  developed  personality  classes  for  aviators.  He  designed  a  series  of 
contrasting  traits  to  distinguish  above-average  versus  below-average  pilot  applicants.  For 
example,  he  preferred  an  “intelligent”  versus  “stupid”  candidate  and  one  who  was  “precise” 
rather  than  “vague.”  He  also  described  personality  characteristics  that  he  perceived  to  be  more 
adaptive  in  the  aviator  environment.  These  traits  included  cheerfulness,  aggressiveness,  and 
attentiveness  [3],  In  1930,  another  flight  surgeon,  Longacre,  also  attempted  to  describe  favorable 
traits.  From  his  perspective,  the  ideal  pilot  would  be  young,  single,  medically  healthy,  educated, 
dextrous,  emotionally  stable,  self-reliant,  intelligent,  and  highly  motivated  [4].  In  assessing  for 
intelligence  and  personality  characteristics,  flight  surgeons  were  attempting  a  more  holistic 
selection  process,  i.e.,  they  were  looking  for  not  just  the  absence  of  medical  or  mental  health 
problems,  but  were  also  interested  in  finding  pilots  who  exhibited  healthy,  adaptive 
characteristics  that  would  enable  them  to  succeed  in  the  demanding  career  field  of  military 
aviation.  Among  American  military  branches,  the  U.S.  Air  Force  (USAF)  contains  the  majority 
of  pilots  and  continues  to  attempt  to  select  pilots  who  are  not  just  medically  healthy  but  also 
intelligent,  emotionally  resilient,  and  highly  motivated  to  fly. 

Within  the  USAF,  there  are  multiple  avenues  to  become  a  pilot.  One  major  dividing  line 
is  between  active  duty  (AD)  and  Air  National  Guard  (ANG)  pilots;  AD  pilots  are  chosen 
primarily  from  USAF  Academy  cadets  and  Reserve  Officer  Training  Corps  (ROTC)  cadets. 
Although  there  are  minor  differences  between  the  selection  techniques  of  the  USAF  Academy 
and  ROTC,  they  are  generally  very  similar  in  their  approach.  The  ANG,  however,  utilizes  a 
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much  more  diverse,  subjective  method  of  selecting  pilots  for  aircraft.  This  is  not  surprising, 
given  the  ANG  is  organized  into  individual  units  located  in  each  of  the  50  states,  U.S.  territories, 
and  Washington,  DC.  Unlike  AD  forces  who  have  one  Commander-in-Chief  (the  President), 
ANG  units  are  controlled  by  state  governors.  Another  important  difference  is  that  ANG  pilots 
are  chosen  for  a  particular  airframe  that  is  flown  by  their  individual  unit,  whereas  AD  pilots 
compete  to  be  a  top  pilot  in  training  to  earn  their  choice  of  aircraft.  This  paper  will  discuss 
similarities  and  differences  between  ROTC  and  ANG  selection  processes  and  compare  the 
intellectual  and  personality  characteristics  of  the  pilots  who  are  deemed  to  have  the  “right  stuff.” 

ROTC  chooses  pilots  through  the  process  of  a  selection  board  that  uses  standardized 
criteria.  This  board  judges  among  the  applicants  using  multiple  standardized  domains.  The  most 
heavily  weighted  domain  is  the  applicant’s  Pilot  Candidate  Selection  Method  (PCSM)  composite 
score.  Carretta  found  the  PCSM  composite  score  was  moderately  correlated  with  completion  of 
T-6  training  in  USAF  pilots  [5].  The  PCSM  score  is  composed  of  three  measurements.  The  first 
is  the  applicant’s  Test  of  Basic  Aviation  Skills  score.  The  Test  of  Basic  Aviation  Skills  is  a 
computerized  test  designed  to  assess  pilot  skills  by  measuring  spatial  orientation  and 
psychomotor  skills  and  multitasking.  The  second  is  the  applicant’s  Air  Force  Officer  Qualifying 
Test  score.  This  is  a  paper  and  pencil  cognitive  test  given  to  all  officers.  The  final  measure  that 
makes  up  the  PCSM  score  is  the  total  number  of  flying  hours  the  applicant  has  accomplished. 
Four  additional  domains  are  used  to  judge  an  applicant’s  suitability  to  become  a  pilot.  The  first  is 
based  on  an  applicant’s  performance  during  field  training  exercises.  The  second  is  the  applicant’s 
perfonnance  during  a  physical  fitness  assessment.  Thirdly,  the  applicant’s  college  grade  point 
average  is  used  to  rank  the  pilot  applicant.  Finally,  a  commander’s  ranking  is  added  to  the 
equation.  The  commander’s  ranking  is  based  on  his  or  her  judgment  of  the  applicant’s  potential 
using  a  “whole  person”  concept.  All  of  these  factors  are  taken  into  account  when  judging  the 
suitability  of  an  ROTC  pilot. 

In  contrast,  the  ANG’s  selection  process  is  not  centralized.  This  allows  for  units 
throughout  the  states  with  different  missions  to  tailor  their  selection  process  to  their  individual 
needs.  Like  ROTC,  ANG  pilot  applicants  are  reviewed  by  a  selection  board.  The  standards  that 
ANG  selection  boards  use  to  select  pilots  vary  by  state  and  can  even  differ  within  states  based  on 
a  commander’s  discretion.  Like  ROTC  selections  boards,  ANG  boards  will  often  use  the  PCSM 
score  as  a  differentiator  among  applicants.  Some  ANG  units  will  have  a  minimum  cut-off  PCSM 
score,  while  others  will  have  less  formalized  standards.  Some  ANG  boards  will  take  into 
consideration  prior  work  history,  motivation  to  fly,  previous  accomplishments,  and  officership 
(i.e.,  leadership).  A  key  difference  between  the  ROTC  and  ANG  selection  process  is  that  ANG 
will  often  attempt  to  hire  from  within  its  ranks.  Enlisted  members,  who  often  are  already  in 
flying  positions  (e.g.,  loadmaster)  and  have  shown  good  leadership  potential,  are  often 
encouraged  to  apply  for  pilot  positions.  The  ANG  has  a  distinct  advantage  over  ROTC  in  that  its 
leadership  will  often  be  able  to  observe  the  performance  of  its  members  over  a  span  of  years 
prior  to  selecting  them  for  a  pilot  slot. 

3.0  METHODS 

Data  were  collected  on  AD  and  ANG  pilots  through  the  Military  Flight  Screening- 
Neuropsychiatry  process,  which  is  baseline  cognitive  and  personality  testing  administered  as  part 
of  the  initial  flight  physical  examination.  This  testing  is  not  a  selection  instrument  but  a  clinical 
tool  that  can  be  used  at  a  later  date  if,  for  example,  the  aviator  sustains  a  traumatic  brain  injury. 
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A  computer-based  test  of  general  intellectual  ability,  the  Multidimensional  Aptitude  Battery-II 
(MAB-II),  and  a  personality  measure,  the  NEO  Personality  Inventory-Revised  (NEO  PI-R),  were 
administered  to  1 1,616  ROTC  and  1,801  ANG  pilot  applicants.  The  MAB-II  is  derived  from  the 
Wechsler  Adult  Intelligence  Scale-Revised  and  is  composed  of  10  subtests.  The  five  verbal 
subtests  include  infonnation,  comprehension,  arithmetic,  similarities,  and  vocabulary.  The  five 
perfonnance  subtests  include  digit  symbol,  picture  completion,  spatial,  picture  arrangement,  and 
object  assembly.  These  subtests  yield  three  composite  scores:  verbal  intelligence,  performance 
intelligence,  and  full-scale  intelligence  (FSIQ).  For  composite  scores,  internal  consistency  (.94 
to  .98)  and  test-retest  reliability  (.92-. 95)  are  reportedly  high  [6]. 

The  NEO  PI-R  is  a  240-item  computer  instrument  that  measures  five  major  domains  of 
personality  [7].  These  include  neuroticism,  extraversion,  openness,  agreeableness,  and 
conscientiousness.  The  NEO-PI-R  also  gives  six  underlying  facets  for  each  domain.  Strong 
evidence  has  been  demonstrated  for  content,  criterion,  and  construct  validity.  Internal 
consistency  (.89  to  .93)  and  test-retest  reliability  (.91  to  .93)  have  been  shown  to  be  high  for 
domain  scores. 

The  mean  age  for  ANG  pilots  was  26  (standard  deviation  (SD)=2. 13),  while  the  mean  age 
for  ROTC  pilots  was  22.97  (SD=2.14).  Of  ROTC  pilots,  90.3%  identified  as  Caucasian,  3.5% 
identified  as  Hispanic,  2.95%  identified  as  Asian/Pacific  Islander,  1.5%  identified  as  Other,  1.3% 
identified  as  African-American,  0.4%  identified  as  Indian  American,  and  0.1%  identified  as 
Arabic.  Demographic  data  for  ANG  pilots  were  not  obtained,  but  King  and  colleagues  reported 
that  in  a  large  sample  of  all  USAF  pilots,  84.2%  identified  as  White,  4.0%  identified  as  Hispanic, 
9.4%  identified  as  Other,  and  2.4%  identified  as  African-American  [8].  For  ANG  pilots,  92.6% 
were  males,  while  93.5%  of  ROTC  pilots  were  males. 

4.0  RESULTS 

Compared  to  the  general  population  (Table  1),  the  mean  score  for  FSIQ  on  the  MAB-II  in 
ANG  pilots  was  at  the  90th  percentile  (mean  (M)=l  18.7,  SD=6.67),  while  ROTC  pilots  had  a 
mean  FSIQ  at  the  92nd  percentile  (M=120.5,  SD=6.32).  ROTC  intelligence  scores  were 
significantly  different  (<p.01)  than  ANG  intelligence  scores  (higher  ROTC  scores),  with  an 
effect  size  of  0.28  (Cohen’s  d).  Figure  1  shows  essentially  all  USAF  pilots  are  above  the  general 
population  mean  of  100. 

Similarities  between  ANG  and  ROTC  pilots  were  also  found  on  personality  domains 
(Figure  2).  Both  ANG  pilots  (M=44.64,  SD=9.09)  and  ROTC  pilots  (M=45.27,  SD=9.37)  were 
below  the  general  population  in  susceptibility  to  emotional  distress  (neuroticism).  Additionally, 
ANG  and  ROTC  pilots  were  higher  than  the  general  population  in  conscientiousness  (M=55.85, 
SD=8.73  and  M=55.78,  SD=9.88)  and  extraversion  (M=58.01,  SD=8.72  and  M=57.94, 

SD=9.54),  respectively.  Significant  differences  between  ANG  and  ROTC  pilots  were  not  found 
for  the  domains  of  extraversion  and  conscientiousness.  However,  ANG  pilots  were  significantly 
different  (p=.01)  on  the  domain  of  neuroticism  (lower  ANG  scores),  with  a  minimal  effect  size 
of  .07  (Cohen’s  d). 
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Table  1.  FSIQ  and  Personality  Scores  of  ANG  Pilots,  ROTC  Pilots,  and  Civilians 


Domain/Facet 

ANG 

ROTC 

Civilian 

Sample 

ANG  vs.  ROTC 

N 

Mean 

SD 

N 

Mean 

SD 

Mean 

SD 

t  value 

Two- 

Sided 

p-value 

Cohen’s 
d  Effect 
Size 

FSIQ 

1,801 

118.7 

6.7 

11,616 

120.5 

6.3 

100.0 

15.0 

11.29 

<0.01 

0.28 

Neuroticism 

1,801 

44.6 

9.1 

11,616 

45.3 

9.4 

50.0 

10.0 

2.67 

0.01 

0.07 

Extraversion 

1,801 

58.0 

8.7 

11,616 

57.9 

9.5 

50.0 

10.0 

0.29 

0.77 

0.01 

Conscientiousness 

1,801 

55.8 

8.7 

11,616 

55.9 

9.9 

50.0 

10.0 

0.28 

0.78 

0.00 

Figure  1.  Box  plot  of  ANG  and  ROTC  FSIQ  scores. 


4 


DISTRIBUTION  STATEMENT  A.  Approved  for  public  release. 


Cleared,  88PA,  Case  #2014-3717. 


Figure  2.  Box  plot  of  ANG  and  ROTC  personality  traits. 


5.0  DISCUSSION 

Although  ANG  pilots  are  selected  using  a  less  standardized  and  more  subjective  method 
than  their  AD  counterparts,  both  processes  produce  pilots  with  favorable  characteristics  for 
success  in  the  aviation  career  field.  Superior  intelligence  has  been  associated  with  success  in 
multiple  career  fields  [9,10]  and  has  long  been  considered  an  important  part  of  the  selection 
process  for  pilots.  The  mean  FSIQ  of  both  groups  of  pilots  is  well  above  the  general  population 
mean,  and  although  ROTC  pilots  had  a  significantly  higher  intelligence  score,  the  difference  is 
not  thought  to  be  meaningful.  Whether  or  not  it  is  necessary  to  have  superior  intelligence  to 
succeed  as  a  military  pilot  is  unclear,  but  anecdotally,  pilots  with  intelligence  well  below  their 
peers  (i.e.,  greater  than  two  standard  deviations)  have  reported  academic  difficulty  in 
Undergraduate  Pilot  Training.  This  may  suggest  that  above-average  intelligence  is  an  important 
aspect  of  making  the  ideal  pilot,  but  that  motivation  and  character  traits,  to  include  neuroticism, 
extraversion,  and  conscientiousness,  also  make  substantial  contributions. 
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Multiple  studies  examining  the  role  of  neuroticism,  extraversion,  and  conscientiousness 
in  military  or  other  dangerous  occupations  have  concluded  that  these  characteristics  play  an 
important  role  in  mission  success  [1 1,12].  ANG  and  AD  pilots  are  low  in  the  character  trait  of 
neuroticism.  People  low  in  neuroticism  are  often  described  as  resilient  and  emotionally  stable. 
These  characteristics  allow  ANG  and  AD  pilots  to  remain  calm  and  steady  when  encountering 
stressful  situations  during  military  flying.  The  trait  also  aids  them  in  juggling  the  demands  of 
work  and  home  life,  and  there  is  evidence  that  USAF  pilots  have  much  lower  rates  of  depressive 
disorders  [13]  and  anxiety  [14]  than  the  general  public.  The  ability  to  compartmentalize,  or  focus 
on  the  present  moment,  is  key  in  dealing  with  the  stressors  of  military  life. 

Interpersonal  skills  play  a  role  in  every  occupation  and  are  known  to  be  an  important 
characteristic  of  successful  pilots.  Another  key  characteristic  for  successful  pilots  is  their  above- 
average  level  of  extraversion.  Extraversion  is  a  combination  of  interpersonal  warmth, 
gregariousness,  assertiveness,  energy  level,  and  openness  to  positive  emotions.  Crew  resource 
management  is  a  component  of  mission  success  for  the  pilot,  and  higher  levels  of  extraversion 
allow  ANG  and  AD  pilots  to  navigate  interpersonal  relationships  with  both  crew  and  staff  while 
also  communicating  accurately  and  decisively  during  combat  missions. 

Finally,  USAF  pilots  are  above-average  in  the  trait  of  conscientiousness.  Siem  &  Murray 
found  that  conscientiousness  was  the  most  important  personality  factor  associated  with  pilot 
success  [15].  Conscientious  individuals  are  achievement  oriented  and  often  have  a  strong  desire 
to  succeed  in  activities  in  which  they  are  engaged.  Pilots  tend  to  be  organized,  dependable,  and 
reliable.  These  are  traits  employers  often  covet  in  employees,  as  they  are  associated  with 
achievement  at  work.  Pilots  who  are  given  the  reins  of  weapons  systems  costing  tens  of  millions 
of  dollars  are  expected  to  be  trustworthy  and  competent,  and  they  are  typically  more 
conscientious  than  their  general  population  peers. 

6.0  CONCLUSION 

Although  ANG  pilots  are  chosen  by  a  more  subjective,  less  structured  process  than  AD 
pilots,  both  groups  have  similar  characteristics  of  high  intelligence,  hardiness,  extraversion,  and 
conscientiousness  that  are  associated  with  success  in  the  aviation  environment. 
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LIST  OF  ABBREVIATIONS  AND  ACRONYMS 


AD 

active  duty 

ANG 

Air  National  Guard 

FSIQ 

full-scale  intelligence 

M 

mean 

MAB-II 

Multidimensional  Aptitude  Battery-II 

NEO  PI-R 

NEO  Personality  Inventory-Revised 

PCSM 

Pilot  Candidate  Selection  Method 

ROTC 

Reserve  Officer  Training  Corps 

SD 

standard  deviation 

USAF 

U.S.  Air  Force 
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